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ABSTRACT 



This paper uses an evaluation of the School Based Youth 
Services Program (SBYSP) , a school - linked, service-integration program in New 
Jersey, to explore how school -community collaborations can achieve both 
educational and social, emotional and behavioral outcomes. The paper provides 
background information on SBYSP and then presents organizational findings 
from the analysis of the project's implementation. The article focuses on 
ways to construct a collaborative working relationship between programs and 
schools and details strategies, such as participating in school committees 
and planning and leading school events, for maintaining good working 
relationships with schools that host the services. Challenges faced by 
school-project collaborations are presented and some lessons about 
collaboration are offered. Profiles of the at-risk student population that 
enrolled in SBYSP are offered, along with an analysis of what the program can 
do for these children. The report gives an overview of findings from the 
evaluation's outcome study, with a focus on educational outcomes associated 
with SBYSP utilization. The text concludes that many collaborative 
school-linked service programs can be an effective strategy for addressing 
students' nonacademic needs. However, school-linked service programs cannot 
be expected to have strong educational outcomes unless they also have strong 
educational components. (RJM) 
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Because of School-Based, I can sleep at night knowing our kids aren 't falling through the 
cracks, [guidance counselor in School Based school] 

The best thing I like about School-Based is that they always offer students the 
opportunity to be involved in positive activities. It is a very productive setting for 
students. School-Based is their second home, [teacher in School Based school] 

Counseling helped me look at myself better and realize I was depressed Now I ’m able to 
talk to my mom about things. They ’ve helped me with my problems and helped me 
concentrate on school. [School Based participant] 



For many advocates of school-linked services and full-service schools, the rationale for these 
arrangements is primarily nonacademic — to address the fragmentation of and inadequate access to 
important services for children in need. These advocates see schools as an excellent delivery site to 
provide services to young people of school age, particularly during the adolescent years. In addition, 
the placement of service providers in schools also makes both prevention and health promotion, as 
well as treatment, possible. Through using the school as a satellite location, service providers 
(whether health, mental health, or employment preparation) can furnish students with an integrated 
array of services in one easy-to-reach location. 

Moreover, as school-linked services and community-school collaborations have increased, 
expectations have grown that these service arrangements can also address noneducational problems, 
usually associated with poverty, that act as barriers to student learning. If services are provided to 
remove these barriers, the reasoning goes, educational outcomes will improve. School-linked 
programs, as a result, face increased demands to produce educational outcomes as the pressure 
intensifies for schools to meet more stringent performance standards. In schools where every 
resource is turned to helping students achieve academically, it has become logical to ask that 
programs using the school as a resource contribute to this push for achievement. 

This paper will use the evaluation of one of the earliest school-linked service integration 
programs — the New Jersey School Based Youth Services Program (SBYSP) — to explore how 
school-community collaborations seek to achieve both educational and social, emotional and 
behavioral outcomes. After providing background on the School Based program, this paper presents 
organizational findings from the analysis of the project’s implementation, followed by an overview 
of findings from the evaluation’s outcome study, with a focus on educational outcomes associated 
with SBYSP utilization. We conclude, based on the experience of the program, that many students 
have substantial nonacademic needs and that collaborative school-linked service programs can be an 
effective strategy for addressing these needs. However, we also conclude that school-linked services 
programs cannot be expected to have strong educational outcomes unless they also have strong 
educational components. 
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The School Based Youth Services Program 



In 1987, the New Jersey Department of Human Services initiated the School Based Youth Services 
Program, the first statewide initiative in the country to integrate a range of services for adolescents 
in one location at or near schools. By creating partnerships between schools and community 
agencies, the program sought to provide young people with the services and supports they needed 
to navigate the adolescent years and “complete their education, obtain skills leading to employment 
or additional education, and lead a mentally and physically healthy life.” 1 With ongoing help from a 
School Based support team in the Department of Human Services, the projects began their first full 
year of operation in 1988 in 29 New Jersey communities, all selected based on high levels of need. 
There is at least one project in every county of the state, and all students attending the host schools 
are eligible to participate in activities and use the services provided. The program has been cited 
repeatedly as an early model of service integration and has won prestigious national awards for 
excellence in public policy. 

Projects are operated by a lead or managing agency, which receives the state grant. The basic 
SBYSP model, as articulated in the 1987 request for proposals, has five core areas of activities and 
services: recreation, health, mental health, employment counseling and preparation, and substance 
abuse treatment and prevention. Services are offered throughout the school year and during the 
summer. Some activities meet regularly, such as peer leadership groups, but other services and 
activities operate on a drop-in or as-needed basis (e.g., recreation or counseling). Many projects have 
added components over time, such as pregnancy prevention and supports for teen parents; conflict 
resolution and violence prevention workshops; peer leadership development and cultural awareness 
activities; and, less frequently, academic support and college visits and efforts to combat stereotyping 
and discrimination based on race, gender and sexual orientation. 

As a whole. School Based services and activities are designed to treat existing problems, prevent the 
emergence of negative youth behaviors, and promote positive youth development. The model has 
evolved substantially since the initiative was launched, although individually-oriented services remain 
central to the work of most projects. In interviews conducted as part of the evaluation of School 
Based, some project directors described their mission as helping students confront barriers to 
succeeding in school, but more often, they focused either on more immediate concerns (for example, 
coping with crises or acute distress) or supporting adolescent development more generally. The 
primary focus was on helping individual youth — “whatever it takes to help the student” in the words 
of the SBYSP program director. 

The Academy for Educational Development’s Evaluation of SBYSP 

In 1995, the Annie E. Casey Foundation, in consultation with the New Jersey Department of Human 
Services, selected the Academy for Educational Development (AED), an educational evaluation and 
technical assistance organization with offices in Washington D.C. and New York City, to conduct 
an evaluation of SBYSP. The AED evaluation is being conducted in two phases. The first phase of 



1 New Jersey Department of Human Services, Request for Proposals, 1 987. 
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the evaluation included an analysis of the state policy context for developing, implementing, and 
sustaining the School Based initiative and a cross-site analysis of program implementation at the site 
level. Evaluators visited every site twice, interviewing project directors and key staff, lead agency 
coordinators, school principals, guidance counselors, nurses and teachers. 2 In addition, at each site, 
site visitors observed activities and conducted a focus group with students involved in the project. 3 

The second phase of the evaluation, which began in summer 1996, is an outcome-based study of the 
program in six individual sites. The strategies for this phase are designed to increase our 
understanding of how individual projects operate, as well as their impact on the young people who 
use them. 4 They include the collection of longitudinal quantitative data via a confidential student 
survey and qualitative data through student interviews and focus groups. 

Before discussing the outcome study and its findings, it is important to understand how the School 
Based programs worked in the school setting to help individual students. Using the data gathered 
at all 29 sites during the implementation study, the following section discusses the evolution of the 
relationship between the projects and their host schools and the challenges encountered in developing 
and maintaining good collaborative working arrangements. 

Findings from the Implementation Study: 

Constructing a Collaborative Working Relationship Between Programs and Schools 

From the beginning of School Based, the New Jersey Department of Human Services has fostered 
a bottom-up collaboration between local schools and local service providers, funded and supported 
from the state level. Although other state-level departments (e g., health, employment) backed the 
new effort, the New Jersey Department of Education had other educational priorities when SBYSP 
was initiated and thus provided neither political nor fiscal support for the new projects. In several 
wary districts, this lack of initial support slowed the acceptance of the program. Nevertheless, 10 of 
the original lead agencies were school districts, and school districts are the most common lead 
agencies (followed by mental health agencies) of School Based projects. 

Several sites encountered opposition to SBYSP based on school personnel’s fear that SBYSP was 
a first step toward privatization and “contracting out” for services currently provided by school 
district employees. In these sites, SBYSP was implemented only when the coordinator for SBYSP 



2 Those teachers we interviewed were recommended by the project directors, at our request, as those having 
knowledge of and contact with the project. 

3 AED produced two reports based on this phase of the evaluation: The New Jersey School Based Youth 
Services Program: The State Policy Context (1997) and The New Jersey School Based Youth Services Program: An 
Analysis of Implementation (1997). 

4 Throughout this paper, we use the terms “users” to describe students who participated in School Based 
activities or who used School Based services, and “nonusers” to describe students who did not. While we employ 
“users” and “nonusers” for reasons of space, the students referred to are both users of services and participants in 
activities offered by the projects. 
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in the department of human services reassured schools that the projects and their staff would “not do 
anything that the schools, could, would or should do themselves.” This promise, still largely in place, 
has framed the relationship between the projects and the schools in important ways. 



Working relationships with host schools 

In all 29 sites, SBYSP personnel have devoted substantial energy to developing and maintaining 
positive working relationships with the host schools in order to better provide services to students. 
Initially, the focus of attention was directed to assisting students attending the selected schools, 
whether in an individual or group format. Eventually most SBYSP projects also began to work with 
the schools as organizations However, the initial intent of such school-focused efforts was the 
enhancement of the project’s capacity to assist individual students. 

In addition to providing a wide range of services and activities aimed at individual students, School 
Based staff also invest considerable effort in becoming a part of the “school family,” as one project 
director put it. As a result. School Based has been integrated into the host schools as project staff 
have built their credibility and demonstrated ways they can help the schools meet the needs of 
students. School Based staff now play a wide variety of roles in most host schools. These roles serve 
to keep the school informed about School Based activities and to keep project staff informed about 
the school. The most frequent activities that School Based staff take part in include: 

• Participating on numerous school committees, such as the principal’s cabinet, the child- 
study and crisis management teams, and the committees charged with attendance review, 
conflict resolution, behavior and discipline, and school safety, as well as a variety of other ad 
hoc committees 

• Planning and executing school events, such as freshman orientation activities, alcohol- and 
drug-free post-prom and graduation parties, and other activities, such as a monthly “positive 
discipline” drawing, a collection of holiday presents for needy children, and a food drive 

• Conducting classes, workshops, and in-service sessions for both students and teachers on 
topics such as the negative impact of stereotyping, contraception, HIV and AIDS, depression 
and anxiety, sexual harassment, and dealing with disruptive students 

• Advocating for and supporting special groups of students, such as teen parents and 
special education students, including conducting parenting classes and support groups, and 
providing health care for mothers and infants, tutoring, and on-site child care (or facilitating 
links to community-based care) 

• Providing substance abuse prevention and crisis management activities, including 
counseling and classes on anger management and conflict resolution and addressing 
schoolwide tragedies and crises 
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In a few cases, SBYSP staff also have assumed traditional school roles or responsibilities, as in one 
site where the recreation coordinator is also a sports coach and the senior class advisor. The projects’ 
capacity to reach students of all kinds is greatly enhanced by the breadth of roles played by School 
Based staff 

SBYSP staff continually work to make their presence widely known throughout the school, 
undertaking numerous outreach activities, such as participating in student and faculty orientation 
meetings; distributing School Based materials to parents; hosting social events for students to make 
them aware of SBYSP offerings; communicating regularly with the principal; and maintaining 
ongoing and informal contact with school staff, students, and parents. Focus groups with students 
and interviews with project staff and school personnel revealed that, as a result of these varied 
outreach activities, students come to School Based in a number of different ways. Some come in 
response to referrals or recommendations from guidance staff, the school nurse, teachers, 
administrators, custodial and security staff, and parents. Other students come with friends or are 
drawn into a School Based activity, such as recreation or a special trip. Still others “self-refer” based 
on their knowledge of School Based and need of assistance. Lastly, in several schools, students 
caught fighting are mandated to work with the SBYSP team on anger management skills. 



Challenges faced by the school-project collaborations 

In a very few cases, host schools have resisted any role for the School Based project beyond 
providing services and activities to students and have made even that difficult. In most cases, 
however, there is abundant evidence of the projects’ integration into the schools in the ways described 
above. Still, School Based projects face certain inevitable challenges in working with the schools, 
given the historic wariness of school staff about outside or “non-school” people, especially regarding 
the issue of professional credentials. 

School Based has sometimes been viewed as a “dumping ground” for problem youth: “I used to think 
this was a place for the nut cases, but you can find just about anything you want here,” explained a 
student who had been referred to School Based for counseling after his father’s death. Not 
surprisingly, the projects with the most balanced array of services and activities are least likely to be 
labeled as a place only for students with problems. This, in turn, enables them to reach the broadest 
range of students. 

In some schools, School-Based projects have been seen as a place for students to hang out and cut 
classes. To counter this misperception, projects make efforts to respect the school’s need to account 
for students’ whereabouts and teachers’ insistence that students not miss class. To deter students 
from cutting classes, most projects have instituted the policy that students can come to the SBYSP 
office during class only with a pass. 

School Based projects have also had to deal with persistent “turf” issues. Given the focus of most 
SBYSP projects on counseling and personal support, these problems most often arise with guidance 
personnel and school nurses over such questions as which personnel are most fit to address students’ 
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needs and how the information about students being seen by SBYSP clinical staff should be handled. 
Despite these concerns in a few schools, evaluators found that most nurses and guidance staff were 
among the strongest supporters of School Based. 

The most problematic turf issue arises from different rules and norms concerning confidentiality and 
information-sharing. While health and mental health professionals (whose norms and practices 
dominate SBYSP on this issue) are governed by medical confidentiality considerations, school staff 
traditionally are less concerned with the protection of individual students’ privacy than with their 
broad responsibility for assuring the well-being of students, both individually and collectively. As a 
result, school staff are more likely to discuss sensitive information openly. However, the assurance 
of student confidentiality is critical to a successful SBYSP project since many students will not reveal 
personal concerns if they think anyone else will learn about their situation. Students in focus groups 
volunteered that they value the confidentiality of School Based services and were quick to say, "What 
we say in here does not leave the room." To maintain confidentiality, most projects have worked out 
mutually satisfactory arrangements with school staff concerning the exchange of information. In 
general, some information is shared — for example, the guidance office is told that a student is being 
seen but not what the assistance concerns. 

Just as there is “positive” turf that some school personnel were reluctant to cede, evaluators also 
found “negative” turf or “turf dumping.” This involves areas that school personnel are sometimes 
too ready to hand over to School Based staff. For example, schools have sometimes requested that 
SBYSP staff handle areas — such as the reporting of suicide and abuse to legal authorities and 
counseling special education students — that are the school’s legal responsibility. 

The role of SBYSP in crisis management activities provides an example of the sometimes murky 
boundaries and complex relationships involved in working with the host schools. School Based staff 
regularly deal with crises that arise in the lives of individual students, working with students and their 
families to secure assistance and address urgent needs. However, in crises that affect the whole 
school, decisions about consequences for student behaviors (e.g., bringing weapons into the school) 
and reporting to legal authorities remain the responsibility of the schools. SBYSP staff frequently 
assist the school in organizing their capacity to respond to schoolwide crises and spearhead the 
provision of assistance to individual students affected by the crisis (e.g., the suicide of a fellow 
student). Unfortunately, taking on responsibilities in schoolwide crisis management can consume 
considerable time and energy, particularly in large and troubled schools. This can lead the School 
Based project to be so reactive that it has difficulty addressing other project goals, such as prevention 
education and support for positive youth development. At the same time, these activities sometimes 
bring School Based staff into contact with youth who might not otherwise know of the project, and, 
as a result. School Based staff often assume at least partial responsibility for these activities. 

Lessons about collaboration 

The effort that most SBYSP projects invest in developing activities in collaboration with school staff 
has helped many projects become integrated into the life of the school, as described above, and avoid 
the “we-they” stance that sometimes characterizes school-community collaborations in their early 
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stages. Despite occasional turf issues, most School Based projects have managed to maintain 
effective working relationships with their schools. This is a labor-intensive process, requiring 
substantial and virtually continuous effort on the part of School Based project directors and their 
staff. More often than not, the collaborative relationships described above were initiated by SBYSP 
staff, and demand a commitment of time beyond that already required to address the needs of 
individual students. The key is ongoing communication: one project director described how she deals 
with boundary and confidentiality issues with her school — "negotiation, negotiation, negotiation.” 

At the same time, it is important to note that the arena where most School-Based projects work is 
carefully circumscribed by the promise made at the program’s initiation: that the program would not 
do anything the school could or should be doing. While some projects offer homework help and 
tutoring for students who seek it, as well as overnight college visits to build students’ interest in 
postsecondary education, the bulk of project work — whether with individual students or the school 
as a whole— is generally limited to student support and behavior issues, and rarely ventures into the 
pedagogical arena that is at the heart of the school’s existence . 5 Nor is it likely that some of the host 
schools, still wary about noncertified staff despite years of positive working relationships, would 
easily accept the participation of SBYSP staff in pedagogical discussions. On the project side, the 
SBYSP directors often have more than enough to do just fulfilling what they have already defined 
as their central mission: helping individual students. 

Despite this limitation, the positive impact of SBYSP projects on the school is evident at many sites. 
Teachers interviewed during visits to projects were quick to express appreciation for the counseling 
available to students and relief that there was somewhere to send students in difficulty, while in the 
past the only recourse was often punitive. Teachers and administrators also recognized that School 
Based’s ability to meet students’ personal needs helped free up both teachers’ and students’ attention 
and energy for teaching and learning. Finally, both teachers and administrators reported that the 
school was able to see students in a more holistic fashion than was possible before School Based. 
One superintendent of schools said “School Based has made the school more responsive to individual 
needs. We are now more prone to see kids as individuals and to look at individual cases. Perhaps 
this means we are more humanistic as a result.” 

Findings from the Outcome-Based Study: 

Identifying How the School Based Youth Services Program Benefits Students 

The following section describes the development and execution of the outcome-based study, the 
differences between those students taking advantage of SBYSP services and activities and those who 
had not done so at the beginning of their first year in high school, and the differences between these 
two groups at the time of the follow-up survey, at the end of their second year in high school. 



5 The one partial exception to this involves a fully integrated collaborative pregnancy prevention program 
combining classroom instruction and services to individual students. SBYSP staff and the school s family life teachers 
work together, both in the classroom and in referring and counseling individual youth. This strikingly successful 
collaboration, which has drastically reduced the incidence of teen births in the host school, demonstrates the potential of 
joining forces in a true school-program partnership. 
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To measure the outcomes that students derive from participation in SB YSP activities and/or use of 
SBYSP services, AED has followed the cohort of students who entered ninth grade in September 
1996 for two years. Students in the six SBYSP schools selected for the outcome study completed 
specially designed confidential surveys at . the beginning of their high-school career (fall 1996) and 
again at the end of their sophomore year (late spring 1997). Using the quantitative data, we were 
able to compare outcomes for students who had taken advantage of SBYSP to those who had not, 
controlling for initial differences in students behavior, background and situational characteristics. In 
addition to the surveys, AED collected school data and tracked a small sample of students from each 
school via individual interviews and focus groups. This qualitative data was collected to illuminate 
the dynamic through which SBYSP achieves its results. 

The sites selected for the outcome-based study vary greatly from one another. The six school districts 
include one regional school district encompassing 142 square miles, one citywide vocational district, 
and four local districts ranging from densely urban to a mixture of urban and suburban or urban and 
rural (a municipality that incorporates a rural area with a densely populated urban center). The 
schools include one vocational and five academic high schools, with student populations that range 
from almost entirely Caucasian in two schools to almost entirely African-American and Latino in 
three. According to the New Jersey Department of Education Report Card data for 1996-97, four 
of the schools’ average scores on the on the High School Proficiency Examination (given to all New 
Jersey students in grade 1 1) were below the statewide average score, and in two of these cases, below 
the average scores for comparable schools. 6 In half the schools, the mobility rate, or number of 
students who entered or left the school during the year, exceeded the statewide average of 16 percent 
and in one school, approached 50 percent. However, one of the schools was in the process of 
planning a comprehensive school reform initiative, and only one of the six schools had a dropout rate 
above the statewide average. 

The SBYSP projects included in the outcome study also varied substantially from site to site in terms 
of lead agencies, space, and programming. Their lead agencies included two school districts, one 
community development agency (working in collaboration with a hospital behavioral health 
department), one hospital family planning department, one local employment agency, and one 
community mental health agency. Four sites operated from space within the school building, while 
one site used a trailer in the school’s parking lot, and another conducted most activities in space 
provided in the local armory. Although all included the core components (recreation, health, mental 
health, employment preparation and/or counseling, and substance abuse prevention and treatment), 
some projects had additional site-specific components, such as an on-site health clinic or a mentoring 
program. 

The following discussion draws on the quantitative data collected during the students’ first two years 
in high school. 



6 Source: New Jersey Department of Education, School Report Cards, 1 996-97. School scores on the HSPT 
are compared both to the statewide average and to the average score for the school’s “District Factor Group based on 
the socioeconomic status of residents in the district. 
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Survey development 



During the implementation analysis, AED, in collaboration with SB YSP project directors, articulated 
a theory of action for each SBYSP major area of activity (recreation, health, mental health, 
employment preparation and/or counseling, and substance abuse prevention and treatment, pregnancy 
prevention, teen parent support, violence prevention, academic support and positive youth 
development). For each service or activity area, the theory of action specified which needs program 
designers and implemented were addressing; the activities they put in place to do so; the anticipated 
response of students to these offerings; and desired mid-range outcomes and their indicators. 

These theories of action framed the development of AED’s primary data collection instruments, 
which collect individual background data and document students’ attitudes and behaviors in the broad 
range of SBYSP services and activities and their desired outcomes. Many individual survey items 
were selected or adapted from other instruments used to study youth . 7 In addition, other items were 
developed specifically for this instrument to ensure sufficient coverage of all facets of SBYSP. 

The survey questions were organized into six categories: 

• Background characteristics (gender, race/ethnicity, family composition) 

• Situational characteristics (level of stress, violence, family, adult, and peer support) 

• Personal characteristics (feelings, educational aspirations, educational history) 

• Behavioral characteristics (sexual activity, violence/delinquency, substance abuse) 

• Health-related characteristics (health status, health risks, access to health care) 

• Youth development characteristics (after school and youth development activities) 

In addition, the follow-up survey also included site-specific questions on SBYSP utilization and 
satisfaction. 

In seeking a comparison population, no single school was appropriate, given the diversity of the six 
selected schools for the outcome study, and resources were not available for multiple comparison 
sites. Instead, outcomes of students taking advantage of SBYSP offerings (“users”) were compared 
to the outcomes of students who did not (“nonusers”). However, because those students using 
SBYSP services and/or participating in SBYSP activities tended to be at higher risk for negative 
outcomes than their nonuser peers, the comparison of outcomes for users and nonusers controlled 
for background characteristics associated with higher levels of need, such as family stress, or 
protective factors associated with a lower incidence of risk behavior. These include family support, 
others sources of adult support, and participation in positive youth activities, sometimes referred to 



7 Items were drawn from the Centers for Disease Control and Prevention’s Youth Risk Behavior Study survey, 
the American Drug and Alcohol Survey, and the National Educational Longitudinal Survey, as well as from other studies 
of youth behavior, such as Gary Wehlage’s dropout prevention studies; Mark Weist’s study of mental health in 
Baltimore high school clinics; AED’s Project Choice evaluation; and a WRI, Inc. study of New York City s high school 
health clinics. 
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as “youth assets.” 8 Still, given the extent of School Based’s integration into the school and the way 
the project may have been reflected in the school environment, the reader is cautioned that the 
students who did not use SBYSP services or participate in SBYSP activities may have benefitted, 
directly or indirectly, from the role the project played in many schools. 

A total of 1,509 youth (84%) responded to the baseline survey, which was administered to students 
by AED staff during single regular class periods. 9 At total of 1,205 students took the follow-up 
survey administered at the end of their second year in high school, representing a response rate of 78 
percent; 922 students took both the baseline and the follow-up survey. 

Through collection of SBYSP Level of Service (LOS) data 10 and school data (average daily 
attendance, grade point average, credit accumulation, and transfer and special education status), 
AED staff were able to identify SBYSP users and nonusers and to append school data to students’ 
survey responses. All data were entered into an SPSS database. The results reported below are 
based only on those 922 students who took both the baseline and follow-up surveys. 

How do SBYSP users compare to nonusers? 

A total of 402 (44%) of students had used a SBYSP sendee or activity at some point during their first 
or second year in high school; 520 (56%) had not used any type of SBYSP service during this period. 
The analysis of the baseline survey revealed important differences between those students who had 
taken advantage of SBYSP activities and those who had not. 

Tables la - le show baseline information for the 922 students in the six schools that took both the 
baseline and follow-up surveys, disaggregated by whether or not they had used any SBYSP services 
or participated in any SBYSP activities by the end of their second year in high school. These baseline 
data verify what the practitioners had long suspected: the students they served on a regular basis were 
at greater risk than the rest of the student body. 



8 The Search Institute has conducted extensive research showing that the presence of these personal and 
community assets act as a protective factor against youth engaging in risk behaviors. See P. Benson, The Troubled 
Journey: A Portrait of 6th- 12th Grade Youth (Minneapolis, MN: Lutheran Brotherhood, 1 990) and E. Roehlkepartain 
and P. Benson, Healthy Communities, Healthy Youth (Minneapolis, MN: Search Institute, 1 996), pamphlet. See also 
L. W. Gregory, “The “Turnaround” Process: Factors Influencing the School Success of Urban Youth.” Journal of 
Adolescent Research. 10(1), 1995. 

9 In the case of special education students with limited reading ability, the survey was sometimes given 
separately in a double-length period. Given the sensitive nature of some questions, the survey was not given to special 
education students who needed the questions read aloud. The survey also was translated into Spanish and administered 
by a bilingual evaluator. 

10 This is the client utilization and management information system developed specially for SBYSP. 
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Background characteristics 



Table la shows the background characteristics of students who participated in SBYSP activities or 
used SBYSP services during their first and second years of high school and those who did not take 
advantage of any SBYSP offerings. 

Table la. Demographic characteristics of SBYSP Users and Nonusers at Baseline (early freshman 





Users 


Nonusers 


Total 


USER STATUS 


402 


520 


922 


DEMOGRAPHICS 








Gender 








Female 


59.7% 


50.8% 


54.7% 


Male 


40.3% 


49.2% 


45.3% 


Race 








American Indian 


1.5% 


1.0% 


1.2% 


Black or African American 


37.7% 


23.8% 


29.9% 


Puerto Rican 


9.7% 


7.1% 


8.3% 


Other Latino/a or Hispanic 


8.0% 


6.7% 


7.3% 


Asian American 


1.2% 


1.3% 


1.3% 


White or Caucasian 


33.9% 


51.0% 


43.5% 


Other or mixed race 


8.0% 


9.0% 


8.6% 



As shown in this table: 

• More than half of SBYSP users were female students (59.7%); in contrast, the nonusers were 
more evenly divided by gender. 

• About one-third (37.7%) of SBYSP users were African American, 33.9 percent Caucasian, 8.0 
percent Latino/a, and 8.0 percent identified their racial/ethnic background as “other.” The 
nonusers included fewer African American, Puerto Rican, Hispanic, Asian American and other 
non-Caucasian students. This may reflect the greater concentration of students in need among 
non-Caucasian students, as well as the commitment of the projects to serve those students not 
drawn into mainstream school activities. 

Areas of risk and stress 

Table lb examines the baseline status of SBYSP users and nonusers with regard to areas of stress or 
support often associated with variations in risk behaviors. 
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Table lb. Areas of Risk and Stress for SBYSP users and nonusers at baseline (early freshman 



USER STATUS 


Users 

n=402 


Nonusers 

n=520 


Total 

n=922 


FAMILY STRESS* 








3 or more stressors 


32.1% 


20.4% 


25.5% 


no stressors 


30.8% 


41.3% 


36.8% 


FAMILY SUPPORT* 








no support 


3.3% 


3.3% 


3.3% 


some support 


96.7% 


96.7% 


96.7% 


OTHER ADULT SUPPORT* 








no support 


5.4% 


3.4% 


4.2% 


some support 


94.6% 


96.6% 


95.8% 



* Scale definitions: 

Family stress is an additive scale composed of 1 2 individual stressful family events or situations (e.g, divorce, substance abuse problems). 
Family support is an additive scale asking if there is a family member to consult regarding a range of 1 0 positive and negative issues. 

Other adult support is an additive scale asking how many non-family adults could be consulted regarding a range of 10 positive and negative 
issues. 



• Users reported much higher levels of family stress, including divorce, residence in unsafe 
neighborhoods, frequent moving, and financial, drug and alcohol problems. Among users, 32.1 
percent had three or more stress factors on a scale of 12 items, as compared to only 20.4 percent 
of the nonusers. Conversely, 41.3 percent of nonusers reported no stress factors, against 30.8 
percent of the users. 

• Users and nonusers reported identical levels of family support, a scale asking how many family 
members one could consult or talk to about a range of both positive and negative issues. In both 
groups, 3.3 percent reported that they had no one to talk to about any of the issues, while 96.7 
percent reported at least some support on some of the issues. 

• Users reported a slightly lower level of support from other adults in their lives. Among users, 5.4 
percent reported having no adults outside their families to turn to for support, while 3 .4 percent 
of nonusers did so. 



Risk behaviors 

Table lc shows the responses of SBYSP users and nonusers to survey questions that asked about a range 
of risk behaviors and factors at the time of the baseline survey, early in their freshman year. 
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Table lc. Risk behaviors of SBYSP Users and Nonusers at Baseline (early freshman year) 



USER STATUS 


Users 

n=402 


Non users 
n=520 


Total 

n=922 


EVER HAD SEX 


30 . 9 % 


23 . 0 % 


26 . 3 % 


DEFINITELY WANT TO AVOID PREGNANCY 


74 . 9 % 


85 . 0 % 


80 . 6 % 


In the last 2 months. . . 








when having sex, always used contraception to prevent pregnancy 


60 . 9 % 


58 . 3 % 


59 . 6 % 


when having sex, always used condoms to prevent STDS 


63 . 6 % 


59 . 3 % 


61 . 5 % 


felt unhappy, sad or depressed 


18 . 9 % 


11 . 2 % 


14 . 5 % 


felt nervous or tense 


17 . 6 % 


11 . 8 % 


14 . 4 % 


worried too much about things 


25 . 8 % 


16 . 5 % 


20 . 6 % 


felt angry or destructive 


19 . 1 % 


12 . 2 % 


15 . 2 % 


hit someone to hurt them 


34 . 2 % 


28 . 0 % 


29 . 9 % 


thought about killing myself 


22 . 6 % 


18 . 2 % 


20 . 1 % 


got into a physical fight 


27 . 0 % 


20 . 7 % 


23 . 4 % 


damaged someone else's property on purpose 


31 . 9 % 


27 . 5 % 


29 . 4 % 


smoked cigarettes 


30 . 8 % 


23 . 2 % 


26 . 5 % 


drank beer or wine 


37 . 2 % 


36 . 7 % 


36 . 9 % 


used marijuana 


19 . 7 % 


10 . 1 % 


14 . 2 % 



• Regarding attitudes and behaviors around sex, fewer users than nonusers affirmed their intention 
to avoid pregnancy during high school. More users than nonusers reported that they had already 
had sexual intercourse, and among users who were sexually active, more reported consistent 
contraceptive use (including condoms) to prevent pregnancy and consistent condom use to 
prevent sexually transmitted disease. 

• On an emotional level, more users than nonusers reported frequent unhappy, sad or depressed 
feelings, as well as tension and frequent worrying. They were also more likely than nonusers to 
report frequent feelings of anger and destructiveness. 

• More users than nonusers also reported that they had been involved in violent behavior, with 
greater proportions of users than nonusers responding that they had hit someone with the 
intention of hurting within the last year, been involved in a fight, or had deliberately damaged 
property belonging to others. 

• Substantially more users than nonusers reported that they had already experimented with 
cigarettes, alcohol, and marijuana in the two months prior to the survey. 
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Positive youth activities 



Table Id presents the responses of both groups of students with regard to their participation in formal 
and informal activities that may have potential protective benefits against engaging in risk behaviors. 

Table Id. Positive Youth Activities of SBYSP users and nonusers at baseline (early freshman 



USER STATUS 


Users 


Non users 


Total 




n=402 


n=520 


n-922 


POSITIVE YOUTH ACTIVITIES 








In the last 2 months. . . 








no activity 


8.4% 


7.3% 


7.8% 


at least one activity once or twice 


14.3% 


11.5% 


12.7% 


at least one activity three or more times 


77.2% 


81 .2% 


79.5% 



• When asked about the frequency of their participation in 1 1 positive activities during the past two 
months, 8.4 percent of users reported no participation at all in any of these activities, compared 
to 7.3 percent of nonusers. However, 14.3 percent of users reported participating in at least one 
activity once or twice, compared to 11.5 percent of nonusers; and 81.2 percent of nonusers 
reported participating in at least one activity three or more times in the past two months, 
compared to 77.2 percent of users. 



Educational characteristics and behaviors 

SBYSP users also differed from nonusers in their educational characteristics and behaviors, as can been 
seen in Table le. 

• SBYSP users did not differ from nonusers in their average daily attendance (95.9% and 96.2%, 
respectively), but were more likely to be classified as special education students (8.8% versus 
5.7%). 

• In addition, users lagged behind nonusers (2.6 compared to 3.2) in mean grade point average 
(GPA) and in average credits earned for their freshman year (33.1 versus 34.2). 

• Users were more likely than nonusers to have cut class more than once in the first two months 
(1 1.8% versus 9.09%) of their freshman year. Users also were more likely to have received a 
failing grade during that period (41.4% versus 34.8%) or to have been sent to the office for 
disciplinary reasons (16.8% versus 9.8%). 
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Table le. Educational Status and Behaviors of SBYSP users and nonusers at baseline (early 



USER STATUS 


Users 

n=402 


Nonusers 

n=520 


Total 

n=922 


EDUCATIONAL STATUS 








Special education status 


8.8% 


5.7% 


7.0% 


Mean grade point average 


2.6 


3.2 


2.9 


Average credits earned 


33.1 


34.2 


33.7 


EDUCATIONAL BEHAVIORS (within first two months of school) 








Cut (skipped) one or more classes Almost every day 


1.5% 


0.6% 


1.0% 


Once or twice a week 


2.3% 


3.1% 


2.7% 


Once or twice a month 


8.0% 


5.3% 


6.5% 


Only once 


11.0% 


9.3% 


10.1% 


Never 


77.3% 


81.7% 


79.8% 


Received a failing grade 


41.4% 


34.8% 


37.7% 


Suspended from school Zero 


93.7% 


96.9% 


95.5% 


Once 


4.7% 


2.7% 


3.5% 


More than once 


1.6% 


0.4% 


0.9% 


Sent to office for discipline 


16.8% 


9.8% 


12.8% 


EDUCATIONAL ASPIRATIONS 








High school only 


2.8% 


4.1% 


3.5% 


Non-degree professional training program/two-year college degree 


10.8% 


9.6% 


10.1% 


Four-year college degree/master’s degree/ doctorate 


73.7% 


70.2% 


71.7% 


Don’t know 


11.1% 


13.5% 


12.4% 


EDUCATIONAL MOTIVATION 








1 do my schoolwork because: (very true only) 








Doing well in school is important to me. 


73.4% 


78.2% 


76.1% 


1 need to learn to get a good job. 


85.5% 


86.5% 


86.1% 


It will help me get my diploma or GED. 


85.5% 


84.8% 


85.1% 


Finishing my education is important to me. 


90.0% 


90.7% 


90.4% 


HIGH DROPOUT RISK* 


22.8% 


17.0% 


19.5% 



* Scale definitions: 

High dropout risk \s the percentage of students who had three or more of the following characteristics: over-age for grade; grade point average 
below 2.0; skipping one or more classes in the two months prior to the survey; receiving a failing grade in the two months prior to the survey; low 
educational aspirations; and special education status. 
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• Both users and nonusers had high educational aspirations, with nearly three-quarters of both 
groups (73.7% of users and 70.2% of nonusers) expressing the intention to pursue at least a four- 
year college degree. If anything, SBYSP users appeared to be a little more clear about their 
intentions and to expect a slightly higher level of educational attainment. 

• Users and nonusers were quite similar in their educational motivation, with only the following 
small differences: doing well in school was less frequently cited as a motive by users than 
nonusers (73.4% and 78.2% respectively), and users slightly more frequently cited the need to 
get a diploma or GED as the motivation for doing schoolwork (85.5% versus 84.8%). 

Dropout risk 

• To measure dropout risk, AED staff constructed a scale combining baseline survey responses and 
school data with regard to the following risk factors: over-age for grade, GPA below 2.0, 
skipping one or more classes in the previous two months, failing grades in the previous two 
months, low educational aspirations, and special education status. Students with three or more 
factors were considered to be at high risk of dropping out. SBYSP users were at substantially 
greater risk of dropping out than their peers who did not use SBYSP, with almost one-quarter 
of users (22.8%) having three or more risk factors, compared to fewer than one-fifth of nonusers 
(17.0%). 

Overall, the responses to the baseline survey clearly demonstrated that SBYSP is attracting those students 
at greater risk for negative outcomes and most in need of assistance if they are to avoid more serious 
problems in both personal and educational domains. 



Patterns in students’ participation in SBYSP during the study period 

Participation in SBYSP activities or use of SBYSP services grew steadily during the students’ first and 
second years in high school. The responses presented above describe the baseline characteristics for 
students who were SBYSP users by the end of their second year in high school. However, an earlier 
analysis of differences between students who had used SBYSP by the end of their first year and those 
who had not showed sharper differences. 11 This most likely results from the earlier participation in 
School Based by students with more severe difficulties, many of whom were referred by school personnel 
to the project for assistance early in their high school career. Over time, as more students became familiar 
with the projects and their offerings, participation expanded, drawing in students who, though at greater 
risk than those not taking advantage of SBYSP, were not as acutely needy as those who came to SBYSP 
in first year. 

During their first two years in high school, students took advantage of a wide variety of services and 
activities, but relatively few did so on a regular or frequent basis, as can be seen in Table 2. As in single- 
focused, school-based service programs (e.g., school clinics and school-based mental health programs), 



11 The New Jersey School Based Youth Services Program Outcome Study: Baseline Report, AED, 1 998. 
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participation was usually episodic rather than continuous and often short-term. The mean scale scores 
for each service or activity show that students’ participation ranged from a low of slightly more than a 
few times a year to a high of between once or twice per month. 



Table 2. Participation in Different SBYSP Services and Activities 



Activity or service 


Number 
using service 


% of all 

students 

(N=922) 


% of SB 

students 

(N=402) 


Mean frequency 
of use* 


Recreation 


264 


28.6% 


65.7% 


1.27 


Individual counseling 


155 


16.8% 


38.6% 


1.98 


Group counseling 


129 


14.0% 


32.1% 


2.46 


Substance abuse counseling 


54 


5.9% 


13.4% 


2.19 


Sexuality-related services 


75 


81.0% 


18.7% 


1.43 


Teen parent services 


39 


4.2% 


5.0% 


1.60 


Health-related services 


104 


11.3% 


25.8% 


1.13 


E m ploy m ent-related 


74 


8.0% 


18.4% 


1.47 


Tutoring 


52 


5.6% 


12.9% 


2.13 



* Frequency of use was rated on the following scale: 



0 = ‘I've never used this service or activity.' 

1 = *A few times a year” 

2 = 'About once a month’ 

3 = 'Frequently (about twice a month)' 

4 = 'Very frequently (about once a week)’ 



Changes from baseline to the end of the second year in high school 

In total, we looked at 37 academic, social/emotional, and behavioral outcomes for youth participating 
in SBYSP. (See box on next page.) The responses of all students in the cohort to the follow-up survey, 
administered at the end of their second year in high school, showed the same overall worsening pattern 
in many areas that is found for many adolescents in difficulty. 12 (See Table 3a, below.) Although both 
groups reported fewer incidents of delinquent behavior, more users and nonusers reported use of tobacco 
and alcohol. In addition, more users and nonusers reported having had sex, and those who reported being 
sexually active had sharply decreased their use of contraception and condoms to prevent STDs. 

Educational motivation also declined for all students, as did students’ aspirations to complete a four-year 
postsecondary education. (See Table 3b, below.) More users and nonusers reported cutting classes, 



12 J. Dryfoos, Adolescents at Risk Prevalence and Prevention , (New Y ork, NY : Oxford University Press, 

1990). 
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receiving failing grades, and being sent to the office for discipline reasons or being suspended from 
school. 



Youth Outcomes Investigated by SBYSP Evaluation 


Academic Outcomes 


Risk Behaviors 


Educational Aspirations 


Damaging property 


Skipping/cutting class 


Stealing money or things 


Receiving failing grades 


Hitting others 


Getting suspended from school 


Getting into a physical fight 


Getting sent to the office for disciplinary reasons 


Having sex 


Positive educational motivation (e.g. doing well in 
school is important) 


Using condoms to prevent pregnancy 


Grade point average 


Using condoms to prevent STD’s 


Credit accumulation 


Getting pregnant 


Average daily attendance 


Wanting to avoid pregnancy while in school 


Social/Emotional Outcomes 


Smoking 


Feeling happy or pleased 


Drinking beer/wine 


Feeling proud of self 


Drinking liquor 


Feeling excited about the future 


Using marijuana 


Feeling too tired to do things 
Having trouble sleeping 
Feeling unhappy, sad or depressed 
Feeling nervous or tense 
Feeling angry or destructive 


Using other drugs 


Feeling close to or appreciated by a friend 
Feeling like you weren’t going to live very long 
Worrying too much about things 
Having suicidal thoughts 
Positive self-efficacy 
Negative self-efficacy 
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Table 3a. Changes in Risk Behaviors from the baseline (early freshman year) to the follow-up survey (late sophomore year) 




COPY AVAILABLE 



Table 3b. Changes in Educational Status and Behaviors from the baseline (early freshman year) to follow-up survey (late sophomore year) 
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In many outcome areas, without controlling for initial differences, SBYSP users appeared to have lost 
more ground than nonusers. That is, SBYSP users showed even greater change in a negative direction 
than their nonusing peers. For example, more users than nonusers reported increased delinquent acts, 
such as stealing and hitting with the intention of hurting. More sexually active users than nonusers 
reported never using contraception or condoms to prevent STDs than at the baseline survey and fewer 
reported that they used protection every time or most times than had originally done so. The greater 
increase in risky behavior among users at the follow-up survey is not surprising considering SBYSP users 
were more at risk and engaged in more risky behavior than nonusers at the baseline survey. Despite this, 
users appeared to make gains compared with nonusers in a few areas. Specifically, users showed greater 
improvement from the baseline to the follow-up survey than nonusers in average daily attendance, grade 
point average, being sent to the office for discipline, multiple suspensions, and use of tobacco and alcohol. 

However, it is important to recall that the baseline survey revealed important differences between students 
who took advantage of SBYSP services and activities and those who did not. This means that the 
outcomes for users and nonusers cannot be compared fairly without controlling for these differences. 
To address this problem, we conducted multiple and logistic regression analyses on the outcome variables 
controlling for prevalence of the outcome or behavior at the baseline, degree of family stress, presence 
of family support and other sources of adult support, and participation in positive youth activities. This 
set of variables allowed us to control for pre-existing (or baseline) differences in user and nonuser 
behavior, as well as for elements that the literature and research have shown to be protective factors in 
youth development . 13 

Another potential threat to the validity of a user/nonuser comparison is selection bias. It could be argued 
that students who seek out and use SBYSP services may be more motivated to get assistance and less 
isolated and disengaged from the school community than their nonusing peers — and therefore, more likely 
to have positive outcomes than nonusers. This does not seem to be the case however. First, not all users 
of SBYSP are self-referred (an indication of self-selection bias). Many students are referred to SBYSP 
by school counselors, teachers and administrators when there are signs of trouble. For other students, 
SBYSP becomes involved in response to a specific incidence, such as a crisis in the family or fighting at 
school (where participation in an SBYSP anger-management work shop is mandatory). Further evidence 
that users were not a self- selected group whose outcomes are likely to be more positive than their 
nonusing peers are the differences in situational characteristics and at-risk behavior at the baseline survey. 
As discussed earlier, users were more likely to have stressful family situations, less likely to have 
supportive networks outside of their family, and more likely to engage in risky behaviors such as drinking 
alcohol, smoking tobacco, taking drugs, and engaging in unprotected sex. Therefore, we were confident 
that self-selection bias did not pose a substantial threat to the validity of our comparison group. 



13 P. Benson, The Troubled Journey: A Portrait of 6th-I2th Grade Youth (Minneapolis, MN: Lutheran 
Brotherhood, 1 990) and E. Roehlkepartain and P. Benson, Healthy Communities, Healthy Youth (Minneapolis, MN: 
Search Institute, 1 996), pamphlet. See also M. D. Resnick, P.S. Bearman, R. W. Blum, K.E. Bauman, K.M. Hams, J. 
Jones, J. Tabor, T. Beuhring, R.E. Sieving, M. Shew, M. Ireland, L. H. Bearinger, J. R. Udry, "Protecting Adolescents 
from Harm: Findings from the National Longitudinal Study on Adolescent Health,” The Journal of the American 
Medical Association, Vol. 278, No. 10, September 10, 1997. 
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Our hypothesis, then, was that although SBYSP user outcomes were worse at the baseline measure 
compared with nonusers and were likely to continue to decline over time, participation in SBYSP might 
reverse the downward trend or at least lessen the rate of decline. In other words, although users were 
unlikely to show greater improvement in these outcomes than nonusers, participation in SBYSP would 
reduce the gap between users and nonusers. Results of regression analyses conducted on the 37 
outcomes investigated support this hypothesis. For 3 1 of the 37 outcomes, we saw greater improvement, 
or less of a decline, for users compared with nonusers. Further, the following 1 1 outcomes showed a 
positive and statistically significant 14 effect of participating in SBYSP: 

• Damaging, destroying or marking up somebody else’s property on purpose 

• Using contraceptives to prevent pregnancy 

• Using condoms for STD prevention 

• Smoking cigarettes 

• Having trouble going to sleep or staying asleep 

• Feeling angry and destructive 

• Worrying too much about things 

• Feeling unhappy, sad or depressed 

• Having suicidal thoughts 

• Expressing higher educational aspirations 

• Accumulating credits toward graduation 

As shown in Table 4, the statistically significant positive findings for the outcomes listed above indicate 
that the greater a student’s use of SBYSP, the greater his or her improvement (or the smaller the decline) 
in the outcome area, compared with nonusers. In six areas— stealing, skipping or cutting classes, using 
marijuana, getting pregnant, wanting to avoid pregnancy while in high school, and feeling proud of 
oneself — the regressions analyses showed negative, but nonsignificant, effects. 



14 Because we hypothesized that SBYSP would have a positive effect on outcomes, we used a one-tailed 
significance test. 




28 



23 



Table 4: Regression Coefficients (Betas and Odds) for Selected SBYSP Outcomes 
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For each regression model, students’ utilization of SBYSP services, 15 their baseline measure of the 
outcome variable (e.g., frequency of smoking at the baseline survey), presence of family and other adult 
support, and participation in positive youth activities were used to predict the outcome variables (e.g., 
frequency of smoking at the follow-up survey). In addition, differences between users and nonusers at 
the baseline measure indicated that there was an interaction effect between SBYSP utilization and 
baseline outcomes. Therefore, the interaction between utilization and baseline measures of the outcome 
was also used to predict the outcome variables, and its coefficient can be interpreted as the effect of 
SBYSP utilization on the outcome. 

Among these 1 1 statistically significant outcomes, 16 the largest effect size, as measured by the regression 
coefficient betas, was seen for using condoms to prevent STDs. Using contraceptives to prevent 
pregnancy, credit accumulation, and smoking also had larger betas than educational aspirations or 
emotional health-related outcomes. However, statistical significance tests suggest that SBYSP’s effect 
on students’ educational aspirations, credit accumulation and smoking was also strong, with highly 
significant (p< .01, two-tailed) main effects. 

SBYSP utilization was the strongest predictor of the outcome variable in most of these models. 
However, for some outcomes, family stress and family support were stronger predictors. Not 
surprisingly, family stress was a greater predictor of students’ feeling unhappy or depressed, worrying 
too much, and feeling angry or destructive than SBYSP utilization. Family stress was also a greater 
predictor of educational aspirations than SBYSP utilization. Level of family support was the strongest 
predictor of vandalism in this model. 

Level of support from other adults was not a statistically significant predictor of any of the 1 1 outcomes, 
suggesting that it is not as strong a protective factor as receiving support from family members and 
participating in positive youth activities. Another possible explanation is that support from other adults 
works as a protective factor only in the absence of support from family members. In other words, if 
students have support from their families, there may not be “value added” from additional support from 
other adults. Additional analyses need to be' conducted to test this hypothesis. 

These results suggest that those students who took advantage of SBYSP services and activities did 
indeed benefit from them in quite important ways that reduce their risks of a range of negative outcomes 
and increase the probability of positive outcomes. Although most educational outcomes did not show 
statistically significant positive effects, the finding of a significant positive association between credit 
accumulation and program participation is very important. It suggests that SBYSP projects have the 
potential to sustain some quite-at-risk students long enough for them to make concrete progress toward 
completing high school. 



15 Utilization of SBYSP is a scale score that takes into account the number of different services used and the 
frequency with which they were used. 

16 Because the outcome variable for damaging property was a dichotomous variable, we used logistic 
regression to test the model. Logistic regressions yield odds rather than betas, therefore the size of the effect for the 
outcome damaging property cannot be compared to the size of the effect of the multiple regression equation outcomes. 
Therefore, effect size comparisons were made only among the 10 multiple regression models. 
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Methodological challenges to Measuring Educational Outcomes 

Measuring educational outcomes for participants of the SB YSP posed several methodological challenges 
endemic to any comparable study. The following section describes these challenges as they related to 
accurately measuring dropouts, the difficulty of measuring SB YSP’ s effect on keeping students from 
dropping out of school, and the limited ability to measure latent effects of SB YSP. 

“Keep-ins” and dropouts. During most focus groups conducted as part of AED’s evaluation, at least 
one student made it clear that, without SB YSP, he or she would not have stayed in school. “If it wasn’t 
for School-Based, I wouldn’t be in school any more,” said one girl in a rural project. Interviewed project 
staff also described students whose connection to school was tenuous and for whom SB YSP had played 
a critical role in keeping them connected to school. If it is true that SB YSP is responsible for the 
persistence of these students as “keep-ins” (as opposed to dropouts), this is an important phenomenon 
to try to measure and understand. We tried to capture it through the data, experimenting with several 
strategies, but found no methodologically sound way of doing so. 

Unfortunately, not dropping out of school when one might otherwise have done so is a statistical non- 
event, no different in appearance from the continued enrollment of other students who had not entertained 
thoughts of leaving. Given the short time frame of the evaluation (covering only the first 2 years of high 
school), the option of comparing dropout rates for SB YSP users and nonusers was not a practical 
possibility, since almost all students were under the legal age for school-leaving. In fact, even after 
students turn 16, actual dropping out is also often a statistical non-event, since relatively few students 
who have become so detached from school as to stop coming suddenly decide to walk into the guidance 
counselor’s office to officially withdraw. More often, according to the guidance counselors, they simply 
stop coming to school and disappear. 

Searching for other ways to tell whether students taking advantage of SB YSP services and activities were 
less likely to leave school prematurely, evaluators considered comparing the rate of non-return (as a result 
of transferring from the school, being a long-term absentee, or any other reason for no longer being 
included on the class rolls) for students in both groups who were at highest risk of dropping out (students 
with more than three risk factors on the dropout scale) and who had taken the baseline survey. This 
involved identifying students who did not take the follow-up survey because they were no longer in 
school, as distinct from those who were still on the rolls but were absent or missed taking the survey for 
some other reason. 

We encountered two problems in this effort. First, the number of non-returning students who were at 
high risk of dropout was sufficiently low that we questioned the validity of interpreting any difference in 
sample attrition between these two groups. Second, even if differences in non-return were visible, those 
factors most likely to be associated with school dropout are very difficult to distinguish from the 
residential instability experienced by many students in families experiencing social and economic stress. 

Keep-ins are not model students. A second challenge encountered in interpreting the educational 
outcome data involved the apparent lack of change in all but two educational outcomes — educational 
aspiration and accumulation of credits toward graduation. Despite apparent movement in a positive 
direction, we did not find statistically significant association between participation in SBYSP and 
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students’ grade point average, educational motivation, attendance, cutting classes, getting sent to the 
office for discipline or being suspended or receiving failing grades. This suggests that, if it is true that 
SBYSP managed to keep some quite-at-risk students in school, they still were not able to address 
students’ underlying educational problems, nor their emotional and behavioral responses to these 
problems and to school in general. 

Latent effects of SBYSP. A third challenge relates to the time frame of our evaluation, and within 
which youth may show outcomes of their participation in SBYSP. Many evaluations show that 
educational, social/emotional and behavioral outcomes are relatively long-term goals and that a significant 
amount of time is required before outcomes achieve measurable magnitude. For example, youth in the 
Quantum Opportunities Program showed statistically significant effects only after four years of 
participation in the intervention. The SBYSP evaluation was limited by the relatively short duration of 
the study. We were able to track students’ outcomes for only the first two years of their high-school 
career. To truly measure the effect of SBYSP on its stated mission of “enabling adolescents, especially 
those with problems, to complete their education, obtain skills that either lead to employment or to 
additional education, and to lead a mentally and physically healthy life,” we will have to continue follow 
this cohort of youth for several more years. 

School-linked Collaborative Services and the Balance Between Social and Educational Outcomes 

In discussing the lessons that can be drawn from the experience of the School Based Youth Services 
Program, this evaluation has looked at two kinds of outcomes: 1) the social, emotional and behavioral 
effects on students using services and/or participating in activities delivered in school settings and 2) the 
academic effects on students taking advantage of these services and activities. Although one might have 
hoped that SBYSP would lead to a greater range improved academic outcomes at statistically significant 
levels, it must be recalled that SBYSP is primarily a school-linked service model, with strong components 
in recreation, health, mental health, employment preparation, and substance abuse prevention, but a 
relatively weak educational component. Indeed, the evaluation’s data have shown that a substantial 
group of students attending these six schools had significant social, emotional and behavioral problems, 
and that SBYSP was reaching those students with the greatest needs. 

Across New Jersey, SBYSP has made substantial contributions to the lives of the students and schools 
it serves. Its nonacademic components have been utilized by thousands of students in all 21 counties of 
the state. Exemplary programs have been implemented in areas such as the prevention of adolescent 
pregnancy and school violence as SBYSP site directors saw these needs arising in their schools. In 
addition, SBYSP has functioned as a magnet for resources, enabling some sites to provide case 
management and child-care services to keep adolescent parents in school. 

Academic improvement of individual students was approached tangentially by most School Based sites. 
The implicit assumption was that when the social, emotional, and behavioral barriers to educational 
achievement were removed, students would be able to achieve academically in ways similar to students 
without these kinds of problems. However, the only academically focused support service provided by 
SBYSP was tutoring. Since tutoring was not one of the core services (being, as it is, on the edge of 
what the school “could, would or should do itself’), it was provided in an inconsistent manner depending 
on funding levels and the integration of the School Based project into the host school. In fact, tutoring 
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was only available in four of the six sites that AED studied in depth, and in only one site on a continuing 
basis. Further, only 12.9 percent of SBYSP students reported receiving tutoring, about one-third the 
number who reported receiving individual or group counseling. Moreover, the average frequency of 
receiving tutoring was only slightly more than once a month. Indeed, an analysis that looked at only 
students using tutoring services on a regular basis might show statistically significant results in a greater 
range of educational outcomes. Unfortunately, our sample size is not sufficiently robust to permit such 
analyses. 

Thus the lack of broad educational outcomes for SBYSP should not be surprising. First, the poverty that 
is associated with the kinds of social, emotional and behavioral problems we observed is also reflected 
in low-performing schools with lower paid teachers. Second, the emotional, familial and educational 
problems that may have caused or contributed to the students’ academic difficulties probably did so long 
before they entered high school. In effect, these students entered high school with academic problems 
that needed to be addressed by more than tutoring. Once students’ personal barriers to academic 
achievement are removed, it does not automatically follow that their academic problems will simply 
disappear or be easily remedied. 

Nevertheless, programs like SBYSP can make major contributions to their host schools. Beyond helping 
students address individual and family problems, school-linked service programs can help create a more 
friendly and supportive environment in a school, where students’ difficulties can be dealt with in a 
supportive and non-punitive manner. They can help schools address their contribution to students’ 
problems (e.g., treating students in a manner that stimulates or reinforces negative personal and academic 
behavior) or develop ways to cope more productively with students whose difficulties lead them to be 
disruptive. The School Based project can also help the school support the transition between middle 
school and high school and help build relationships with parents. Finally, the program has rooted itself 
in the communities where it exists and has made it acceptable for programs within schools to address 
problems once considered outside of school purview, such as pregnancy prevention. In doing so, it has 
helped the community regard the school as a center for needed services to their families, and it has led 
to improvements in the access and delivery of social services. 

In addition to school-linked service collaborations, however, schools need to bring together the 
supportive services and activities that address students nonacademic needs with effective educational 
approaches. Interventions that have achieved impressive educational gains, such as the Quantum 
Opportunities Program or Marshalltown, Iowa’s Caring Connection, have strong educational components 
that involve at-risk students in ongoing educational support activities, as well as a range of social and 
emotional support services and activities. 

To achieve the improved educational outcomes so urgently demanded by today s educational authorities, 
collaborative service programs need to be matched to appropriate educational strategies that address 
students’ academic problems in a major way, like the programs cited above. And it is important not to 
underestimate the labor-intensive effort involved in developing the collaborative relationships that enable 
most educational interventions to work. 
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Discussion 



This last section presents conclusions and recommendations for school-community collaborations seeking 
to improve educational outcomes, based on the experience of the New Jersey School Based program. 

Community-school collaborations that focus on linking the school to vital services would be well 
advised to address the full age-range of young people — from the elementary school level through the 
end of high school. Given the pressures in the lives of today’s children and youth, strong programs are 
needed that can support them at different stages during their development. There is no way of predicting 
at what point young people will need educational and emotional support. For example, a student who 
is doing well, both educationally and emotionally, at the end of elementary school may still confront a 
devastating family crisis, in middle or high school. 

Timeliness of services is also critical. By high school, students’ unaddressed educational problems often 
have sunk deep roots, with emotional and behavioral consequences. Addressing these educational 
problems years after they began often requires much more comprehensive attention than might have been 
needed earlier, including intensive educational support — which may be beyond the scope of a 
collaborative program to offer — as well as accompanying individual attention from the schools. 



Finally, it is important that the push for better educational outcomes not lead to a categorical focus for 
school-linked service programs, narrowing the availability of services to students with academic 
difficulties. While, as a whole, SBYSP users were less solid students than nonusers, many among them 
were functioning reasonably well on an academic level but had serious needs for services. 

However rich the program model, it is critical to have realistic expectations of what a supportive 
community-school collaboration can do, and not promise funders and policy makers more than can be 
delivered, as the following quote makes clear: 

Having realistic bench-marks and negotiating obtainable and measurable outcomes increases 
multifold the possibility for successful outcomes— which increases the possibilities of 

continued funding This is not to seek iow standards or outcome measures which could 

be conceived as having little merit. Rather it is to note that setting unrealistic or 
unobtainable outcomes has a deeper price than just the failure of program or the demise of 
a collaborative arrangement. The real cost is in the loss of services to human beings who 
need them. . . Measurable outcomes, goals, and targets all need to be extremely sensitive 
to this fact.” 17 

Securing official support for collaboration from both sides of the organizational divide is critical 
to both the initial and continuing strength of the partnership. This means not only visible support 
in policy language, but also administrative mandates backed up with funding for technical assistance to 



17 James R. Garvin and Alma H. Young, “Resource issues: a case study from New Orleans,” in The Politics of 
Linking School and Social Services, edited by Louise Adler and Sid Gardner, The 1 993 Y earbook of the Politics of 
Education Association, Washington, D.C., The Falmer Press, 1 994 , p. 1 05. 
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support collaboration, and the inclusion of the “capacity to collaborate with outside organizations” as a 
criterion in procedures for selecting and evaluating school principals and guidance staff. 

School-community collaborations benefit from continued technical assistance and support. The 
experience of SBYSP, with the continuing technical support provided by the New Jersey Department of 
Human Services , 18 as well as that of other community-school collaborations — such as the New York City 
Beacons Initiative (assisted by the Youth Development Institute of the Fund for the City of New 
York) — show that such programs benefit enormously from the ongoing provision of resources, technical 
assistance, and networking opportunities. These resources sustain staff who cope daily with the 
challenges of addressing students needs and developing and maintaining collaborative working 
relationships with the host school, local service providers and the surrounding community. 

In the case of New Jersey, the School Based support team provides ongoing access to information, 
training, and other resources, as well as site-level intelligence on how to cope with the challenges of 
constructing both effective School Based programming and good collaborative relationships with schools, 
lead agencies, and community-based agencies. In addition, the monthly School Based meetings, 
organized by the support team, provide project leaders the opportunity to fully articulate the vision of 
the collaboration and learn ways to narrow the gap between the vision and the reality. By providing 
funds for sufficient time and the technical and networking support to achieve success, the School Based 
initiative laid the groundwork for public support and policy changes to support the initiative. 

Community-school collaborations, however well designed and implemented, cannot substitute for 
reform of the schools, nor can a program succeed if it is simply an add-on to a failing school: 

Interdisciplinary cooperation, no matter how expert it might be, cannot solve systemic 
breakdowns. It is a short step from this observation to the realization that interagency 
collaboration efforts are doomed to failure if they are merely “pasted on” to an existing 
system which is failing to establish professional control over basic school program 
implementation . 19 

Efforts to address students’ social, emotional, and behavioral needs must be accompanied by equally 
committed efforts to improve the schools these students attend. Edward Tetelman, one of the creators 
of New Jersey’s School-Based Youth Services Program, has appeared before the state legislature and 
worked within bureaucratic channels to push for increased funding for the schools served by SBYSP 
projects, challenging the legislature, in so many words, to do what SBYSP, under the best of 
circumstances, can never be expected to do — improve the schools: 



18 The role of the support team at the New Jersey Department of Human Services is discussed in The New 
Jersey School Based Youth Services Program: The State Policy Context Title , Academy for Educational Development, 

1 998. The organizational location of the capacity building intermediary agent is less important than its ability to work 
with the projects, both individually and collectively, in a constructive and nonthreatening manner. 

19 Douglas E. Mitchell and Linda D. Scott, “Professional and institutional perspectives on interagency 
collaboration” in The Politics of Linking School and Social Services, edited by Louise Adler and Sid Gardner, The 1 993 
Yearbook of the Politics of Education Association, Washington, D.C., The F aimer Press, 1 994 , p. 84. 
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While we can begin to reduce negative social factors and help a child become ready to learn, 
we cannot, in fact, move the learning process if it is not understandable, interesting, or 
challenging for the youngsters. We must address how children and youth are taught and 
make serious changes on that side of the equation. . . We must do both — provide social 
service supports and alter the learning side if we are to see real long-term investment .” 20 



20 Edward Tetelman, Assistant Commissioner and Director of the Office Legal and Regulatory Affairs, New 
Jersey Department of Human Services, Testimony delivered before the Education Funding Review Commission, August 
18 , 1993 . 
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